A least-squares functional for joint exit wave reconstruction and image registration

Images generated by a transmission electron microscope (TEM) are blurred by aberrations from the objective lens and can be difficult to interpret correctly. One possible solution to this problem is to reconstruct the so-called exit wave, i.e. the electron wave in the microscope right before it passes the objective lens, from a series of TEM images acquired with varying focus. While the forward model of simulating a TEM image from a given exit wave is known and easy to evaluate, it is in general not possible to reconstruct the exit wave from a series of images analytically. The corresponding inverse problem can be formulated as a minimization problem, which is done in the well known MAL and MIMAP methods. We propose a generalization of these methods by performing the exit wave reconstruction and the registration of the image series simultaneously. We show that our objective functional is not convex with respect to the exit wave, which also carries over to the MAL and MIMAP functionals. The main result is the existence of minimizers of our objective functional. These results are based on the properties of a generalization of the cross-correlation. Finally, the applicability of our method is verified with a numerical experiment on simulated input data

Joint denoising and distortion correction of atomic scale scanning transmission electron microscopy images

Nowadays, modern electron microscopes deliver images at atomic scale. The precise atomic structure encodes information about material properties. Thus, an important ingredient in the image analysis is to locate the centers of the atoms shown in micrographs as precisely as possible. Here, we consider scanning transmission electron microscopy (STEM), which acquires data in a rastering pattern, pixel by pixel. Due to this rastering combined with the magnification to atomic scale, movements of the specimen even at the nanometer scale lead to random image distortions that make precise atom localization difficult. Given a series of STEM images, we derive a Bayesian method that jointly estimates the distortion in each image and reconstructs the underlying atomic grid of the material by fitting the atom bumps with suitable bump functions. The resulting highly non-convex minimization problems are solved numerically with a trust region approach. Well-posedness of the reconstruction method and the model behavior for faster and faster rastering are investigated using variational techniques. The performance of the method is finally evaluated on both synthetic and real experimental data

Time Discrete Geodesic Paths in the Space of Images

In this paper the space of images is considered as a Riemannian manifold using the metamorphosis approach, where the underlying Riemannian metric simultaneously measures the cost of image transport and intensity variation. A robust and effective variational time discretization of geodesics paths is proposed. This requires to minimize a discrete path energy consisting of a sum of consecutive image matching functionals over a set of image intensity maps and pairwise matching deformations. For square-integrable input images the existence of discrete, connecting geodesic paths defined as minimizers of this variational problem is shown. Furthermore, $\Gamma$-convergence of the underlying discrete path energy to the continuous path energy is proved. This includes a diffeomorphism property for the induced transport and the existence of a square-integrable weak material derivative in space and time. A spatial discretization via finite elements combined with an alternating descent scheme in the set of image intensity maps and the set of matching deformations is presented to approximate discrete geodesic paths numerically. Computational results underline the efficiency of the proposed approach and demonstrate important qualitative properties.Comment: 27 pages, 7 figure

A Posteriori Error Control for the Binary Mumford-Shah Model

The binary Mumford-Shah model is a widespread tool for image segmentation and can be considered as a basic model in shape optimization with a broad range of applications in computer vision, ranging from basic segmentation and labeling to object reconstruction. This paper presents robust a posteriori error estimates for a natural error quantity, namely the area of the non properly segmented region. To this end, a suitable strictly convex and non-constrained relaxation of the originally non-convex functional is investigated and Repin's functional approach for a posteriori error estimation is used to control the numerical error for the relaxed problem in the $L^2$-norm. In combination with a suitable cut out argument, a fully practical estimate for the area mismatch is derived. This estimate is incorporated in an adaptive meshing strategy. Two different adaptive primal-dual finite element schemes, and the most frequently used finite difference discretization are investigated and compared. Numerical experiments show qualitative and quantitative properties of the estimates and demonstrate their usefulness in practical applications.Comment: 18 pages, 7 figures, 1 tabl

Joint methods in imaging based on diffuse image representations

This thesis deals with the application and the analysis of different variants of the Mumford-Shah model in the context of image processing. In this kind of models, a given function is approximated in a piecewise smooth or piecewise constant manner. Especially the numerical treatment of the discontinuities requires additional models that are also outlined in this work. The main part of this thesis is concerned with four different topics. Simultaneous edge detection and registration of two images: The image edges are detected with the Ambrosio-Tortorelli model, an approximation of the Mumford-Shah model that approximates the discontinuity set with a phase field, and the registration is based on these edges. The registration obtained by this model is fully symmetric in the sense that the same matching is obtained if the roles of the two input images are swapped. Detection of grain boundaries from atomic scale images of metals or metal alloys: This is an image processing problem from materials science where atomic scale images are obtained either experimentally for instance by transmission electron microscopy or by numerical simulation tools. Grains are homogenous material regions whose atomic lattice orientation differs from their surroundings. Based on a Mumford-Shah type functional, the grain boundaries are modeled as the discontinuity set of the lattice orientation. In addition to the grain boundaries, the model incorporates the extraction of a global elastic deformation of the atomic lattice. Numerically, the discontinuity set is modeled by a level set function following the approach by Chan and Vese. Joint motion estimation and restoration of motion-blurred video: A variational model for joint object detection, motion estimation and deblurring of consecutive video frames is proposed. For this purpose, a new motion blur model is developed that accurately describes the blur also close to the boundary of a moving object. Here, the video is assumed to consist of an object moving in front of a static background. The segmentation into object and background is handled by a Mumford-Shah type aspect of the proposed model. Convexification of the binary Mumford-Shah segmentation model: After considering the application of Mumford-Shah type models to tackle specific image processing problems in the previous topics, the Mumford-Shah model itself is studied more closely. Inspired by the work of Nikolova, Esedoglu and Chan, a method is developed that allows global minimization of the binary Mumford-Shah segmentation model by solving a convex, unconstrained optimization problem. In an outlook, segmentation of flowfields into piecewise affine regions using this convexification method is briefly discussed

Mesh-to-raster based non-rigid registration of multi-modal images

Region of interest (ROI) alignment in medical images plays a crucial role in diagnostics, procedure planning, treatment, and follow-up. Frequently, a model is represented as triangulated mesh while the patient data is provided from CAT scanners as pixel or voxel data. Previously, we presented a 2D method for curve-to-pixel registration. This paper contributes (i) a general mesh-to-raster (M2R) framework to register ROIs in multi-modal images; (ii) a 3D surface-to-voxel application, and (iii) a comprehensive quantitative evaluation in 2D using ground truth provided by the simultaneous truth and performance level estimation (STAPLE) method. The registration is formulated as a minimization problem where the objective consists of a data term, which involves the signed distance function of the ROI from the reference image, and a higher order elastic regularizer for the deformation. The evaluation is based on quantitative light-induced fluoroscopy (QLF) and digital photography (DP) of decalcified teeth. STAPLE is computed on 150 image pairs from 32 subjects, each showing one corresponding tooth in both modalities. The ROI in each image is manually marked by three experts (900 curves in total). In the QLF-DP setting, our approach significantly outperforms the mutual information-based registration algorithm implemented with the Insight Segmentation and Registration Toolkit (ITK) and Elastix

On the structure of defects in the Fe7Mo6 μ\mu-Phase

Topologically close packed phases, among them the $\mu$-phase studied here, are commonly considered as being hard and brittle due to their close packed and complex structure. Nanoindentation enables plastic deformation and therefore investigation of the structure of mobile defects in the $\mu$-phase, which, in contrast to grown-in defects, has not been examined yet. High resolution transmission electron microscopy (HR-TEM) performed on samples deformed by nanoindentation revealed stacking faults which are likely induced by plastic deformation. These defects were compared to theoretically possible stacking faults within the $\mu$-phase building blocks, and in particular Laves phase layers. The experimentally observed stacking faults were found resulting from synchroshear assumed to be associated with deformation in the Laves-phase building blocks

Joint non-rigid image registration and reconstruction for quantitative atomic resolution scanning transmission electron microscopy

Aberration corrected scanning transmission electron microscopes (STEM) enable to determine local strain fields, composition and bonding states at atomic resolution. The precision to locate atomic columns is often obstructed by scan artifacts limiting the quantitative interpretation of STEM datasets. Here, a novel bias-corrected non-rigid registration approach is presented that compensates for fast and slow scan artifacts in STEM image series. The bias-correction is responsible for the correction of the slow scan artifacts and based on a explicit coupling of the deformations of the individual images in a series via a minimization of the average deformation. This allows to reduce fast scan noise in an image series and slow scan distortions simultaneously. The novel approach is tested on synthetic and experimental images and its implication on atomic resolution strain and elemental mapping is discussed

A distribution-dependent Mumford-Shah model for unsupervised hyperspectral image segmentation

Hyperspectral images provide a rich representation of the underlying spectrum for each pixel, allowing for a pixel-wise classification/segmentation into different classes. As the acquisition of labeled training data is very time-consuming, unsupervised methods become crucial in hyperspectral image analysis. The spectral variability and noise in hyperspectral data make this task very challenging and define special requirements for such methods. Here, we present a novel unsupervised hyperspectral segmentation framework. It starts with a denoising and dimensionality reduction step by the well-established Minimum Noise Fraction (MNF) transform. Then, the Mumford-Shah (MS) segmentation functional is applied to segment the data. We equipped the MS functional with a novel robust distribution-dependent indicator function designed to handle the characteristic challenges of hyperspectral data. To optimize our objective function with respect to the parameters for which no closed form solution is available, we propose an efficient fixed point iteration scheme. Numerical experiments on four public benchmark datasets show that our method produces competitive results, which outperform two state-of-the-art methods substantially on three of these datasets